# Lightweight Transformer

SAUTE is a lightweight Transformer architecture with speaker perception ability, designed for effectively modeling multi-speaker dialogues.

Terjman-Nano-v2.0 is a Transformer-based English-Moroccan dialect translation model with 77M parameters, optimized for high-quality and precise translation.

Spec-Vision-V1 is a lightweight, state-of-the-art open-source multimodal model designed for deep integration of visual and textual data, supporting a 128K context length.

A distilled version of the genmoai mochi-1 model transformer, composed of 42 modules (original version has 48 modules), achieving lightweight through iterative removal of modules with the smallest MSE values

A text classification model fine-tuned based on microsoft/Multilingual-MiniLM-L12-H384, used to classify email subjects as spam (SPAM) or non-spam (NOSPAM).

A lightweight semantic segmentation model based on the Segformer architecture, pre-trained on the Cityscapes dataset

Sapiens is a family of vision transformers pre-trained on 300 million 1024 x 1024 resolution human images for depth estimation tasks.

State-of-the-art sentence segmentation technology using a 3-layer Transformer architecture, supporting multilingual text segmentation.

sat-3l is a model suitable for wtpsplit, adopting a 3-layer Transformer architecture to achieve state-of-the-art sentence segmentation.

MeshGPT is a text-to-3D model based on autoencoders and Transformers, the world's first publicly available 3D model tokenizer.

Octo Small is a diffusion policy model for robot control, based on Transformer architecture, capable of predicting robot actions from visual inputs and language instructions.

A sentence embedding model based on sentence-transformers that maps text to a 384-dimensional vector space, suitable for semantic search and text similarity calculation

This is a French sentence embedding model based on DistilCamemBERT, capable of encoding sentences or paragraphs into 768-dimensional vectors for tasks such as sentence similarity computation.

Simple Stories is a series of small text generation models trained on the TinyStories dataset, focusing on generating children's stories.

Octo Small is a robot control model trained based on diffusion policy, capable of predicting 7-dimensional actions for the next 4 steps, suitable for multi-source robot datasets.

CED is a simple audio tagging model based on ViT-Transformer, achieving state-of-the-art performance on Audioset.

A lightweight pre-trained model based on Transformer architecture, specifically designed for bidirectional translation between Vietnamese Nôm and Latin script

A semantic segmentation model based on the MobileViTv2 architecture, pre-trained on the PASCAL VOC dataset, supporting 512x512 resolution image processing

SegFormer is an efficient semantic segmentation model based on Transformer, with the b0 version being its lightweight implementation.

This model is a fine-tuned version of wav2vec2-base-ft-keyword-spotting on the superb dataset for audio classification tasks, optimized through quantization and structured pruning.

An image feature model based on Vision Transformer (ViT), trained using the self-supervised DINO method, suitable for image classification and feature extraction tasks.

A lightweight pre-trained encoder-decoder Transformer model designed for Vietnamese news summarization

State-of-the-art lightweight pretrained model for Vietnamese based on Transformer encoder-decoder architecture, specialized for text summarization tasks.

NAT-Mini is a lightweight vision Transformer model based on neighborhood attention mechanism, designed for ImageNet image classification tasks

T5-small is a pre-trained model based on the encoder-decoder architecture, capable of handling multiple tasks through a unified text-to-text format, supporting multilingual processing.

LeViT-128S is a vision Transformer model pretrained on the ImageNet-1k dataset, combining the advantages of convolutional networks for faster inference.

LeViT-384 is a vision Transformer model pre-trained on the ImageNet-1k dataset, combining the advantages of convolutional networks for faster inference speed.

A sentence representation model for semantic retrieval compressed via homomorphic projection distillation, with only 23 million parameters and a model size of 87MB

A text classification model fine-tuned on the GLUE COLA dataset based on google/fnet-base, used to evaluate the performance comparison between FNet and BERT architectures

This is a sentence embedding model based on sentence-transformers, capable of mapping sentences to a 768-dimensional vector space, suitable for tasks such as semantic similarity calculation and text clustering.

XtremeDistilTransformers is a knowledge-distilled lightweight Transformer model with task-agnostic properties, applicable to various natural language processing tasks.

MiniLM is a compact and efficient pre-trained language model, compressed through deep self-attention distillation technology, suitable for language understanding and generation tasks.

DeiT is an efficiently trained vision Transformer model, pretrained and fine-tuned on the ImageNet-1k dataset, suitable for image classification tasks.

MiniLM is a compact and efficient pre-trained language model that compresses Transformer models through deep self-attention distillation technology, supporting multilingual understanding and generation tasks.

This is a truncated MT5-base model with vocabulary and word embeddings optimized for Chinese and English characters, suitable for Chinese-English text processing tasks.

XtremeDistilTransformers is a task-agnostic Transformer model distilled through task transfer learning, creating a small universal model applicable to any task and language.

XtremeDistilTransformers is a task-agnostic distilled Transformer model that utilizes task transfer learning techniques to train small general-purpose models suitable for various tasks and languages.

DeiT is a more efficiently trained Vision Transformer model, pre-trained and fine-tuned on the ImageNet-1k dataset at 224x224 resolution, suitable for image classification tasks.

DistilRoBERTa is a distilled version of the RoBERTa-base model with fewer parameters but faster speed, suitable for English text processing tasks.

This is a distilled version of distilbert-base-multilingual-cased, specifically supporting English and Urdu while maintaining the original model's representation capabilities.